From M/G/1 and M/G/infinity to Simulation: Staged Service Performance Design

There are various techniques for dealing with server saturation. Under duress, some variation of admission control is usually applied, including possibly refusing (aka “throttling”) all arrivals. As parallelism, threading, etc. become commonplace, alternative approaches to servicing heavy workloads are feasible. For a given stream of arriving requests, classical M/G/1 and M/G/infinity queuing models can establish initial estimates for both serial and parallelized service requirements. As analysis and design proceed, request streams that are not necessarily exponential are understood more fully. Verified simulation can subsequently translate queuing model predictions to more refined workload service allocations. Such staged modeling is also extendable to capacity planning for arbitrary service workloads.